Potential uses of AI for perioperative nursing handoffs: a qualitative study

Abstract Objective Situational awareness and anticipatory guidance for nurses receiving a patient after surgery are keys to patient safety. Little work has defined the role of artificial intelligence (AI) to support these functions during nursing handoff communication or patient assessment. We used interviews to better understand how AI could work in this context. Materials and Methods Eleven nurses participated in semistructured interviews. Mixed inductive-deductive thematic analysis was used to extract major themes and subthemes around roles for AI supporting postoperative nursing. Results Five themes were generated from the interviews: (1) nurse understanding of patient condition guides care decisions, (2) handoffs are important to nurse situational awareness, but multiple barriers reduce their effectiveness, (3) AI may address barriers to handoff effectiveness, (4) AI may augment nurse care decision making and team communication outside of handoff, and (5) user experience in the electronic health record and information overload are likely barriers to using AI. Important subthemes included that AI-identified problems would be discussed at handoff and team communications, that AI-estimated elevated risks would trigger patient re-evaluation, and that AI-identified important data may be a valuable addition to nursing assessment. Discussion and Conclusion Most research on postoperative handoff communication relies on structured checklists. Our results suggest that properly designed AI tools might facilitate postoperative handoff communication for nurses by identifying specific elevated risks faced by a patient, triggering discussion on those topics. Limitations include a single center, many participants lacking of applied experience with AI, and limited participation rate.

communication on likely problems and generally understand the patient. Our participants stated that if AI identified likely issues, they would discuss those topics in handoff, communicate about those problems with physicians, and modify their monitoring and treatment to the level of risk faced by the patient. This finding runs against most research on improving communication, which focuses on fixed checklists of topics to discuss. Most uses of AI for nurses focus on making specific to-do recommendations and documentation reminders, but we find that nurses would benefit from AI which focuses more on their understanding of the patient's condition.

BACKGROUND AND SIGNIFICANCE
Inpatient handoffs are the transfer of responsibility, information, and control between clinicians or teams. Incomplete or inaccurate handoffs are a source of subsequent medical errors and patient injury, 1-3 particularly for patients undergoing major surgery. [4][5][6][7] We focus on postoperative nurse handoffs during surgical patient transfers from the operating room (OR) to the postanesthesia care unit (PACU) and from the PACU to inpatient ward. Handoffs are important for receiving nurses to understand the patient's situation because residual sedation, pain, delirium, fatigue, and surgical injuries can make patient-nurse communication difficult. Additionally, the patient's context changes; surgery eliminates some concerns and creates the opportunity for new complications. The data surrounding surgical patients are voluminous and diverse while simultaneously incomplete, which strains the ability of receiving nurses to review and assimilate it de novo. [8][9][10] Two functions of handoff are of special interest to us: situational awareness and anticipatory guidance. Situational awareness is the combination of perceiving critical factors in the environment, understanding what those factors mean for the clinician's goals, and understanding what will happen next. 11 Anticipatory guidance is the communication of likely patient status changes and plans for how to address them. 12,13 These 2 functions support early recognition and coordinated treatment of complications, which have substantial effects reducing postoperative mortality and morbidity. 14 Major handoff quality improvement projects have integrated both of these concepts. [15][16][17] Protocols and checklists are employed to ensure that key information is transmitted during handoffs throughout healthcare. [18][19][20] Some electronic health records (EHRs) have integrated standardized handoffs, 21 including nurse-to-nurse handoffs 15,22 and perioperative nursing handoffs specifically. 23,24 Nevertheless, handoff-related information gaps are common for postoperative patients. 10,[25][26][27][28] The EHR has promise for mitigating and reducing these information gaps. EHRs place an enormous amount of data at the fingertips of all clinicians. In theory, this ought to allow a nurse to prepare for handoff and recover from an incomplete handoff. Dashboard-type displays can be used during handoffs for this summary function. 29 Despite this promise, most handoff-EHR integration work does not focus on the critical functions of situational awareness and anticipatory guidance. 30 Staggers et al 31 found that existing EHR handoff summaries were too rigid and incomplete to be useful; additionally, they interfered with the receiving nurse's encoding of information via note taking. They subsequently found that nurses made little use of EHR handoff support due to these limitations. 32 Calculations and displays of EHR data can be viewed as sense-making, with tension between different purposes and users. 33 Artificial intelligence (AI) integrated into EHRs is an exciting, related development. AI is a broad term, including all computer programing which replicates or imitates cognitive functions. The most common approach applying AI to EHR data for nursing is supervised machine learning (ML), in which algorithms use EHR data as inputs to predict unknown or unrecorded characteristics of a patient, such as future adverse events, current patient condition, or undocumented comorbidities. 34 Although often discussed exchangeably, ML (an approach to pattern recognition) and clinical decision support (CDS) (applying pattern recognition to suggest actions or documentation) are conceptually different. For a given AI/ML pattern recognition tool, a wide variety of uses cases, visualizations, and user interfaces are possible. AI using EHR data has become much more general and accurate in the last few years, 35,36 allowing prediction of perioperative events [37][38][39][40][41] and learning effective treatment strategies. 42 AI is able to interpret nursing documentation to recognize patient types and predict clinical deterioration. [43][44][45][46][47] Research has explored AI/ML in several roles to augment the capabilities of bedside nurses, including identifying care needs or predicting adverse events based on EHR data, scheduling and equipment management, patient activity tracking, processing nursing documentation for transitions of care, quantifying risks in family discussions, and interactive patient education. 34,[48][49][50] For example, ML identification of patients with a high risk of pressure ulcers 51,52 or falls 53 can trigger CDS for nursing interventions. The related CDS literature for nurses has focused on recommending specific actions based on scoring systems and expert-devised rules. 54 In addition to predicting adverse events, AI/ML models can flag important data for review. While information dashboards have long been integrated into EHRs with expert-driven rules for abnormal data, 31,32,55-57 contemporary systems include AI/ML models to identify "relevant" patient data. [58][59][60] Very few AI studies have gone beyond initial development phases or shown benefits to stakeholders, 49,50 and the more developed usecases are often highly specialized, such as rapid-response-team alarms. 48 Expanding nursing engagement in design of AI projects is a recognized priority, 61 as very few AI or information system studies involve nurses at early stages. 50,62 A handful of studies have considered the impact of AI prediction in augmenting handoff communication. In the neonatal ICU context, Hunter et al 63 used natural-language generation to summarize EHR data and generate potential problems and care plans in a dynamic shift-change report. Forbes and colleagues 56,64 envisioned a dynamic EHR integrated shift-report summary for nurses including key data, diagnoses, and predicted adverse events. Hunter and Forbes's work 56,63,64 suggests a distinct role for AI prediction from traditional CDS: facilitating problembased report and assessment during handoffs. Although clinician assessment of the patient's condition is a key part of all structured handoffs, AI identification of likely complications and important data integrated into dynamic "handoff sheets" could supplement handoff assessment more flexibly than traditional checklist-based protocols.
We previously explored related ideas at the OR to intensive care unit handoff, which often has a brief nurse-to-nurse component due to the multidisciplinary nature of the handoff. 65,66 Key findings of that study were the difficulty of making EHR information universally accessible, the need to focus on AI with direct relevance to patient care, and general acceptance of blending AI risk prediction with current summaries of patient data into a handoff tool.
However, the ICU shift-change and OR-ICU handoffs previously studied are quite different from the OR-PACU-ward transition.

OBJECTIVES
Although direct experimentation with implementing AI support for perioperative handoffs would be informative, we set out to establish a use-case with clinicians and refine what content would be useful for clinicians prior to implementation. We identified 3 unanswered preliminary questions in prior research about postoperative bedside nurses as givers or receivers of handoff which we aim to address: (1) would postoperative nurses accept AI recommendations for handoff topics? (2) would nurses find AI-based predictions of adverse events useful and relevant? (3) would a single presentation of AI-based predictions be acceptable to most nurses? The goal of this single-center qualitative study was to explore these topics and how AI added to a handoff workflow might fit into the situational awareness, assessment, monitoring, and communication goals of postanesthesia care unit (PACU) and postoperative ward nurses. We intend these findings to guide subsequent design and implementation efforts, but we did not evaluate a specific AI product or technical implementation.

MATERIALS AND METHODS
Our research included 2 activities: direct observation of handoffs to establish context in the research team and interviews with postoperative nurses to directly address the research questions.

Setting
Barnes-Jewish Hospital is a 1400-bed academic medical center in St Louis, Missouri. We focused on the Acute and Critical Care Surgery (ACCS) division, which performs approximately 1600 inpatient surgeries annually, primarily trauma, and acute abdominal surgery. All postoperative patients (other than those directly admitted to intensive care) recover from anesthesia in the PACU, a 30-bed area. Four hospital units subsequently care for ACCS patients: 2 dedicated hospital wards and 2 high-dependency units. The high-dependency units are shared with otolaryngology, abdominal organ transplant, and hepatobiliary services.

Observations
Researchers selected surgical cases for direct observation from the OR schedule based on the primary surgery service (ACCS). We also included patients likely to be admitted to high-dependency units based on their procedures. We attempted observation on all cases meeting these criteria between 9 AM and 5 PM on weekdays. Researchers conducted direct observations under Washington University IRB approval (#201812137 and #202009066) with the consent of the PACU nurse to shadow their interactions with other clinicians (OR circulator nurse, anesthesia clinician, surgery clinician, and wards nurse) and recorded notes following a structured outline. 67 The IRB approved verbal consents with electronic provision of study information as a replacement for written consents during the coronavirus disease 2019 pandemic. Because we performed these observations to provide interpretative context for interview analysis rather than directly answer study questions, we do not separately report findings from observations. We include this description only to report the nurse participant recruitment process.
Description of perioperative handoff processes and care teams Figure 1 illustrates the handoff process. Prior to surgery, a preoperative holding area nurse completes a health status inventory in the Epic EHR and on a paper record (Supplementary Appendix S1) which is passed to PACU. The preoperative nurse and OR circulating nurse complete an informal handoff. After surgery, a surgery resident or fellow, the OR circulating nurse, and an anesthesia clinician transport the patient to PACU. OR to PACU handoff follows a protocol (Supplementary Appendix S1), where the circulating nurse, surgeon, and anesthetist each give handoff to the PACU nurse. The handoff sheet (Supplementary Appendix S1), consent documents, and backup records from surgical implants, and blood transfusions are the only common paper records. All other documentation is electronic.
Once PACU staff and the supervising anesthesiologist deem a patient ready to leave the PACU, the PACU nurse gives handoff to the ward nurse either at the bedside (high dependency unit) or by phone call (ward units). A guideline addresses the handoff between PACU and the wards nurses (Supplementary Appendix S1). Fellows, resident physicians, nurse practitioners, and the attending surgeon jointly manage postoperative patients. The nurse practitioner or resident physician implementing ward care is not directly involved in the surgery. We refer to that resident or nurse practitioner as the midlevel clinician.

Interview participants and data collection
Concurrently with our direct observations, we recruited a convenience sample of nurses from the PACU, ACCS wards, and highdependency units. We chose interviews instead of focus groups to allow us to hear multiple independent perspectives, and for pragmatic reasons. During the study period, nurse participants faced high workloads, making scheduling focus groups difficult. We conducted interviews under Washington University IRB approval (#201812137 and #202009066) with the consent of the participant. Authors King and Shambe conducted interviews using the same guide (Supplementary Appendix S2). The content of the interviews focused on handoff communication, patient assessment, physician communication, and potential roles for AI. We conducted interviews over the phone or voice application with audio recording, which was transcribed verbatim.

Analysis
Two researchers (King and Shambe) double-coded interviews using a mixed inductive-deductive reflexive thematic analysis approach. First, we familiarized ourselves with the data by reviewing the transcripts and fragmenting them into topical sections. Second, 68 we Preop nurse assessment recorded to paper, provided to circulating nurse. No protocol.
Intraop team (anesthesia, surgery, circulator) gives report and preop nursing sheet to PACU nurse. Detailed protocol. PACU nurse gives phone or bedside handoff to wards nurse. Preop sheet included in paper chart. Partial protocol. organically generated open codes after the first review. We applied deductive codes based on relevance to major study questions (listed in Supplementary Appendix S3). We labeled each statement as relevant to OR-PACU or PACU-ward handoffs based on the surrounding context. Next, the coders discussed the set of open codes and resolved conflicts by consensus. We generated initial subthemes from groups of related codes. We then compared OR-PACU and PACU-ward coded data for similar subthemes that could be coalesced. We did not formalize a codebook, but we returned to the raw statements for consistency with the subthemes and examined them for relationships to other identified subthemes. We then jointly refined subthemes based on recoded data and clustered subthemes into themes based on connecting stories. At each stage, coders compared codes and resolved disagreements. The coders and a third researcher (Abraham) reviewed and revised themes. After the construction of the coding tree, coders checked statements to validate their applicability to the higher-level themes. After 10 interviews, we completed a first round of coding, and we found that most topics were addressed by multiple participants, meaning that saturation was likely; we found no new topics during analysis of the 11th interview and stopped recruitment.
Supplementary Appendix S4 is a consolidated criterion for reporting qualitative research (COREQ) checklist, 69 a qualitative research reporting framework, along with some additional methods details.

RESULTS
We conducted 11 total interviews: 7 PACU nurses and 4 ward nurses. Supplementary Table S5 (Supplementary Appendix S5) displays the 5 major themes in our findings, subthemes, and exemplar quotations of each subtheme: (1) nurse understanding of patient condition guides care decision; (2) handoffs are important to nurse situational awareness, but multiple barriers reduce their effectiveness; (3) AI may address barriers to handoff effectiveness; (4) AI may augment nurse care decision making and team communication outside of handoff; and (5) EHR user experience and information overload are likely barriers to using AI during handoffs. These themes had substantial interactions, and with each subtheme, we note closely related subthemes. Supplementary Table S5 shows the relevance to OR-PACU, PACU-ward, or both handoffs of each subtheme along with number of interviews referencing each.

Nurse understanding of patient condition guides care decisions
Participants stressed that their bedside presence allowed rapid detection and hopefully mitigation of complications. They universally agreed that their understanding of the issues facing a patient modified what signs and symptoms they were alert for (Subtheme 1.b), what issues they communicated to the PACU or midlevel clinician (Subtheme 1.c), and what treatments they recommended. Several participants stated that although almost all treatment changes required a team discussion, their recommendations were likely to be considered or acted on.
Handoffs are important to nurse situational awareness, but multiple barriers reduce their effectiveness Participants stressed that accurate handoff was a critical way to learn about the patient's state, expectations for recovery, and needs in the high-turnover environment of PACU (Subtheme 2.a).
However, they acknowledged barriers where the documentation they relied on was incomplete (Subtheme 2.d), the handoff-giver did not know the relevant information, or they did not understand what needed to be conveyed. Participants agreed that problem-focused handoffs with anticipatory guidance were extremely useful, but that many topics in handoffs were not relevant or recited data without context (Subtheme 2.b). Closely related to this concern was a lack of shared priorities between the handoff giver and receiver. It was frequent for participants to describe receiving handoffs focusing on details they found to be irrelevant or unintelligible, and for handoff, participants to not value topics on which their counterparty asked questions (Subtheme 2.c).

AI may address barriers to handoff effectiveness
Several participants commented on how AI risk prediction at handoff might mitigate mismatch between handoff givers and receivers. First, almost all participants agreed that if AI identified a patient at high risk for a complication, that this topic would be prioritized for discussion at handoff, and that those receiving handoff would ask follow-up questions regarding the patient state and the current plan (Subtheme 3.a). Second, a high calculated risk could alert them that a known comorbidity was more severe than they expected (Subtheme 3.b), which was information frequently absent from documentation. Third, awareness that a patient was overall high-risk would prompt nurses to closely review all available data and prioritize shared careful patient evaluation (Subtheme 3.c). Finally, automatic identification of EHR data elements which increased the patients' risk could mitigate data omissions, especially if that data was in an unusual location (Subtheme 3.d). Although several participants gave examples of how they might relate data given at handoff to specific AI-identified problems (ameliorating the laundry-list type handoff of Subtheme 2.c), none explicitly identified using the AIidentified problems to organize data.
AI may augment nurse care decision-making and team communication outside of handoff PACU handoff is a critical time for establishing joint plans and midlevel clinician communication needs; however, posthandoff communication was also regarded as important. Ward participants noted that midlevel clinicians rarely proactively contacted them, leaving nurses to deduce what issues required communication or nursing action (Subtheme 4.a). Some participants noted that AI could help target posthandoff nurse-midlevel communication in 2 ways. First, if a patient had been identified as high risk, the resistance to contacting the midlevel clinicians to discuss that topic would be lowered (Subtheme 4.b). Second, the nurse's holistic view of patient risk might be difficult to communicate, and AI-based pattern matching would make this more concrete and easier to request midlevel clinicians act on or personally evaluate.
Participants noted incomplete midlevel clinician documentation and other EHR information negatively affected their independent assessment of the patient (Subtheme 4.c). AI identification of alternative key data would then be valuable. Additionally, AI-identified risks for adverse events would allow the nurse to better target their assessment and monitoring independent of any effect on handoff (Subtheme 4.d). Participants noted that AI-identified elevated risks could allow them to target interventions within their scope of practice, such as fall prevention, delirium prevention, and pneumonia prevention (Subtheme 4.d). Multiple participants endorsed the desire for more accurate prediction of patients likely to require higher nursing workload or ICU transfer, which they could use to allocate their resources.
EHR user experience and information overload are likely barriers to using AI Participants identified several barriers for nursing use of AI, largely centered around the user experience and the potential for excessive information volume. First, because of the large number of different methods for accomplishing most tasks in Epic, participants did not recommend the same locations for viewing AI risk prediction. Second, preferred visualizations also differed between participants, with participants variously endorsing absolute risk estimates, relative risks, simplified high-medium-low risk flags, and plots. Several participants noted that existing clinical decision support and alerts already generate alarm fatigue, and that additional flags would likely be ignored unless they had high value (Subtheme 5.c). Finally, participants noted the potential for information overload with more complex outputs (Subtheme 5.d).

DISCUSSION
Our interviews highlighted the importance of team communication and anticipatory guidance at and around postoperative handoffs for nurses to optimize patient care. The data gave consistent answers to our knowledge-gap questions: 1. Would postoperative nurses accept AI recommendations for handoff topics? Yes, participants believed that AI which identified patients at elevated risk would lead to focused handoff communication and physician-nurse team communication on those topics, increasing anticipatory guidance and situational awareness. Nurses overall expressed little hesitance to include AI-estimated risks in their handoff assessments. 2. Would nurses find AI-based predictions of adverse events useful and relevant? Yes, participants believed that well-functioning AI risk assessment would lead to activating nurse-driven interventions, allocating resources (such as high-dependency beds) more efficiently, and prioritizing monitoring for higher-risk outcomes.
To accomplish this, participants desired both overall measures of acuity and estimation of a broad collection of risks. 3. Would a single presentation of AI-based predictions be acceptable to most nurses? No, participants acknowledged diverse methods of using the EHR, and diverse preferences for information presentation. While our participants were enthusiastic for AI identification of relevant information in the EHR, they also acknowledged barriers surrounding the user experience of adding AI to their workflows and the potential for information overload. The ability to easily integrate AI into multiple EHR workflows and choose a personalized presentation will be necessary for it to succeed.
Our work contrasts with much of the development of EHR AI support for nurses, 54 which largely focuses on medication documentation, medication administration, and very simple rule-based systems to identify specific nursing needs. Our work also highlights the need for handoff communication to adapt to the patient's condition, contrasting with the dominant theme of the literature for improving handoffs: standardized communication and checklists. 70 Several small studies from other nursing contexts have found similar themes. Home care nurses in a prior study expressed a similar use case for AI to modify the intensity of their services but did not discuss its role in transitions of care. 71 User-design work for EHR-integrated shift-change handoff support had similar ideas, arriving at a design which blended data and predictive risks. 56,64 Although their work stemmed from interactions with nurses and nursing students, their manuscripts do not give enough methods details to further explore similarities with our work. Nurse users largely accepted a prototype system for shift change in the neonatal intensive care unit which focused on summarizing data in natural language and included expert decision rules as a minor component. 63 Our findings can also be related to work with dashboards intended to detect change in patient status which lack explicit AI predictions. 54 In our work on OR to ICU handoffs, 66 participants endorsed similar desires to integrate AI into summaries of patient data like laboratory results and vital signs and the need to focus on actionability. In contrast to ICU participants, our participants felt that AI augmentation of handoff topics could be useful, AI assessment of risks for midlevel clinician communication would be valuable, and that AI could assist their selection of necessary patient assessment steps. Very recently, experience with risk-predicting AI suggests that it facilitates a shared mental model and coordination across disciplines by providing a reference point for patient status, 72 including using this shared reference point for escalation of care. 73 Our participants echoed this idea in Subtheme 4.b.
Similar to others, [73][74][75] we found that extraction of directly interpretable patient data and actionable needs was a high priority (Subtheme 4.d). Prior work has also found that nurses more frequently use a "bottom-up" (data and needs first) approach to patient summarization, 76 which agrees with our finding of specific riskincreasing data and conditions being important for handoff support (Subtheme 3.c). Physicians and nurses rate explainability in terms of patient data and personal understanding as highly related to trust in AI; 73,77 however, current methods of AI explainability have been found to have limited usefulness in practice. 78 Some implementation studies have found that AI-based alerts are relatively more salient to nurses than physicians in this regard. 79 Imperative AI-based CDS has been effective in some direct use cases, supporting this approach, 80 but it runs the risk of automation bias. 81,82 Similar to the findings of others, 73 our participants indicated that they would consider the AI as a suggestion of where to start an evaluation rather than a prescriptive mandate (Subtheme 4.d).
Taken together, our findings and these prior studies suggest that AI can support nurses in their more general cognitive tasks, and that future AI design efforts should (1) target critical moments of evaluation like shift change and handoff and (2) incorporate estimates of acuity, condition severity, and influential data outside narrow "nursing related" problems. We anticipate that an adaptive handoff sheet design like Hunter and Forbes's work 56,63,64 containing automated identification of problems relevant to each patient and data pertinent to those problems will emerge from further research with this population and ongoing technical testing. This optimism is restrained by the many practical implementation difficulties that plague clinical AI, 83 which was echoed in the concerns of our participants (Subtheme 5).

Limitations
Our study drew participants from a single center, which limits the range of experiences and exposure to alternative EHRs. The ward nurses worked in a small number of units, limiting the generalizability. The number of participants and recruitment rate from those potentially eligible were both low. The participants had limited experience with AI, which limits the reliability of the findings. The setting was an academic medical center, so the views may not reflect the experiences of those outside this type of setting. Our interview was semistructured, and participants were informed on the nature of our study. They may have endorsed ideas to be agreeable, but participants seemed to feel free to disagree.

CONCLUSION
This interview study of perioperative nurses at an academic medical center found that participants were receptive to AI as a potential adjunct for postoperative handoff communication. Ongoing studies will evaluate the usability and communication impact of AI tools in nursing practice.